With the success of Vision Transformers (ViTs) in computer vision tasks, recent arts try to optimize the performance and complexity of ViTs to enable efficient deployment on mobile devices. Multiple approaches are proposed to accelerate attention mechanism, improve inefficient designs, or incorporate mobile-friendly lightweight convolutions to form hybrid architectures. However, ViT and its variants still have higher latency or considerably more parameters than lightweight CNNs, even true for the years-old MobileNet. In practice, latency and size are both crucial for efficient deployment on resource-constraint hardware. In this work, we investigate a central question, can transformer models run as fast as MobileNet and maintain a similar size? We revisit the design choices of ViTs and propose an improved supernet with low latency and high parameter efficiency. We further introduce a fine-grained joint search strategy that can find efficient architectures by optimizing latency and number of parameters simultaneously. The proposed models, EfficientFormerV2, achieve about $4\%$ higher top-1 accuracy than MobileNetV2 and MobileNetV2$\times1.4$ on ImageNet-1K with similar latency and parameters. We demonstrate that properly designed and optimized vision transformers can achieve high performance with MobileNet-level size and speed.

Localizing anatomical landmarks are important tasks in medical image analysis. However, the landmarks to be localized often lack prominent visual features. Their locations are elusive and easily confused with the background, and thus precise localization highly depends on the context formed by their surrounding areas. In addition, the required precision is usually higher than segmentation and object detection tasks. Therefore, localization has its unique challenges different from segmentation or detection. In this paper, we propose a zoom-in attentive network (ZIAN) for anatomical landmark localization in ocular images. First, a coarse-to-fine, or "zoom-in" strategy is utilized to learn the contextualized features in different scales. Then, an attentive fusion module is adopted to aggregate multi-scale features, which consists of 1) a co-attention network with a multiple regions-of-interest (ROIs) scheme that learns complementary features from the multiple ROIs, 2) an attention-based fusion module which integrates the multi-ROIs features and non-ROI features. We evaluated ZIAN on two open challenge tasks, i.e., the fovea localization in fundus images and scleral spur localization in AS-OCT images. Experiments show that ZIAN achieves promising performances and outperforms state-of-the-art localization methods. The source code and trained models of ZIAN are available at https://github.com/leixiaofeng-astar/OMIA9-ZIAN.

最近，稀疏培训已成为有希望的范式，可在边缘设备上有效地深入学习。当前的研究主要致力于通过进一步增加模型稀疏性来降低培训成本。但是，增加的稀疏性并不总是理想的，因为它不可避免地会在极高的稀疏度下引入严重的准确性降解。本文打算探索其他可能的方向，以有效，有效地降低稀疏培训成本，同时保持准确性。为此，我们研究了两种技术，即层冻结和数据筛分。首先，层冻结方法在密集的模型训练和微调方面取得了成功，但在稀疏训练域中从未采用过。然而，稀疏训练的独特特征可能会阻碍层冻结技术的结合。因此，我们分析了在稀疏培训中使用层冻结技术的可行性和潜力，并发现它有可能节省大量培训成本。其次，我们提出了一种用于数据集有效培训的数据筛分方法，该方法通过确保在整个培训过程中仅使用部分数据集来进一步降低培训成本。我们表明，这两种技术都可以很好地整合到稀疏训练算法中，以形成一个通用框架，我们将其配置为SPFDE。我们的广泛实验表明，SPFDE可以显着降低培训成本，同时从三个维度中保留准确性：重量稀疏性，层冻结和数据集筛分。

内存处理（PIM）是一种越来越多地研究的神经形态硬件，承诺能量和吞吐量改进以进行深度学习推断。 PIM利用大量平行，有效的模拟计算在内存内部，绕过传统数字硬件中数据移动的瓶颈。但是，需要额外的量化步骤（即PIM量化），通常由于硬件约束而导致的分辨率有限，才能将模拟计算结果转换为数字域。同时，由于不完善的类似物到数字界面，PIM量化中的非理想效应广泛存在，这进一步损害了推理的准确性。在本文中，我们提出了一种培训量化网络的方法，以合并PIM量化，这对所有PIM系统无处不在。具体而言，我们提出了PIM量化意识培训（PIM-QAT）算法，并通过分析训练动力学以促进训练收敛，从而在向后传播期间引入重新传播技术。我们还提出了两种技术，即批处理归一化（BN）校准和调整精度训练，以抑制实际PIM芯片中涉及的非理想线性和随机热噪声的不利影响。我们的方法在三个主流PIM分解方案上进行了验证，并在原型芯片上进行了物理上的验证。与直接在PIM系统上部署常规训练的量化模型相比，该模型没有考虑到此额外的量化步骤并因此失败，我们的方法提供了重大改进。它还可以在CIFAR10和CIFAR100数据集上使用各种网络深度来获得最受欢迎的网络拓扑结构，在CIFAR10和CIFAR100数据集上，在PIM系统上达到了可比的推理精度。

视觉变压器（VIT）正在出现，并且在计算机视觉任务中的准确性显着提高。但是，它们的复杂架构和巨大的计算/存储需求对新硬件加速器设计方法施加了紧迫的需求。这项工作提出了基于提议的混合速度量化的FPGA感知自动VIT加速框架。据我们所知，这是探索模型量化的第一个基于FPGA的VIT加速框架。与最先进的VIT量化工作（仅无硬件加速的算法方法）相比，我们的量化在相同的位宽度下可实现0.47％至1.36％的TOP-1精度。与32位浮点基线FPGA加速器相比，我们的加速器在框架速率上的提高约为5.6倍（即56.8 fps vs. 10.0 fps），对于DeitBase的ImagEnet数据集，精度下降了0.71％。

基于深度学习的超分辨率（SR）近年来由于其高图像质量性能和广泛的应用方案而获得了极大的知名度。但是，先前的方法通常会遭受大量计算和巨大的功耗，这会导致实时推断的困难，尤其是在资源有限的平台（例如移动设备）上。为了减轻这种情况，我们建议使用自适应SR块进行深度搜索和每层宽度搜索，以进行深度搜索和每层宽度搜索。推理速度与SR损失一起直接将其带入具有高图像质量的SR模型，同​​时满足实时推理需求。借用了与编译器优化的速度模型在搜索过程中每次迭代中的移动设备上的速度，以预测具有各种宽度配置的SR块的推理潜伏期，以更快地收敛。通过提出的框架，我们在移动平台的GPU/DSP上实现了实时SR推断，以实现具有竞争性SR性能的720p分辨率（三星Galaxy S21）。

视觉变压器（VIT）显示了计算机视觉任务的快速进步，在各种基准上取得了令人鼓舞的结果。但是，由于参数和模型设计的数量大量，例如注意机制，基于VIT的模型通常比轻型卷积网络慢。因此，为实时应用程序部署VIT特别具有挑战性，尤其是在资源受限的硬件（例如移动设备）上。最近的努力试图通过网络体系结构搜索或与Mobilenet块的混合设计来降低VIT的计算复杂性，但推理速度仍然不令人满意。这导致了一个重要的问题：变形金刚在获得高性能的同时可以像Mobilenet一样快吗？为了回答这一点，我们首先重新审视基于VIT的模型中使用的网络体系结构和运营商，并确定效率低下的设计。然后，我们引入了一个尺寸一致的纯变压器（无需Mobilenet块）作为设计范式。最后，我们执行以延迟驱动的缩小，以获取一系列称为EfficityFormer的最终模型。广泛的实验表明，在移动设备上的性能和速度方面，有效形式的优势。我们最快的型号，EfficientFormer-L1，在ImagEnet-1k上获得$ 79.2 \％$ $ TOP-1的准确性，仅$ 1.6 $ MS推理潜伏期在iPhone 12上（与Coreml一起编译），该{运行速度与MobileNetV2 $ \ Times Times 1.4 $（ $ 1.6 $ MS，$ 74.7 \％$ top-1），我们最大的型号EfficientFormer-L7，获得了$ 83.3 \％$精度，仅$ 7.0 $ MS延迟。我们的工作证明，正确设计的变压器可以在移动设备上达到极低的延迟，同时保持高性能。

Color fundus photography and Optical Coherence Tomography (OCT) are the two most cost-effective tools for glaucoma screening. Both two modalities of images have prominent biomarkers to indicate glaucoma suspected. Clinically, it is often recommended to take both of the screenings for a more accurate and reliable diagnosis. However, although numerous algorithms are proposed based on fundus images or OCT volumes in computer-aided diagnosis, there are still few methods leveraging both of the modalities for the glaucoma assessment. Inspired by the success of Retinal Fundus Glaucoma Challenge (REFUGE) we held previously, we set up the Glaucoma grAding from Multi-Modality imAges (GAMMA) Challenge to encourage the development of fundus \& OCT-based glaucoma grading. The primary task of the challenge is to grade glaucoma from both the 2D fundus images and 3D OCT scanning volumes. As part of GAMMA, we have publicly released a glaucoma annotated dataset with both 2D fundus color photography and 3D OCT volumes, which is the first multi-modality dataset for glaucoma grading. In addition, an evaluation framework is also established to evaluate the performance of the submitted methods. During the challenge, 1272 results were submitted, and finally, top-10 teams were selected to the final stage. We analysis their results and summarize their methods in the paper. Since all these teams submitted their source code in the challenge, a detailed ablation study is also conducted to verify the effectiveness of the particular modules proposed. We find many of the proposed techniques are practical for the clinical diagnosis of glaucoma. As the first in-depth study of fundus \& OCT multi-modality glaucoma grading, we believe the GAMMA Challenge will be an essential starting point for future research.

Focusing on the complicated pathological features, such as blurred boundaries, severe scale differences between symptoms, background noise interference, etc., in the task of retinal edema lesions joint segmentation from OCT images and enabling the segmentation results more reliable. In this paper, we propose a novel reliable multi-scale wavelet-enhanced transformer network, which can provide accurate segmentation results with reliability assessment. Specifically, aiming at improving the model's ability to learn the complex pathological features of retinal edema lesions in OCT images, we develop a novel segmentation backbone that integrates a wavelet-enhanced feature extractor network and a multi-scale transformer module of our newly designed. Meanwhile, to make the segmentation results more reliable, a novel uncertainty segmentation head based on the subjective logical evidential theory is introduced to generate the final segmentation results with a corresponding overall uncertainty evaluation score map. We conduct comprehensive experiments on the public database of AI-Challenge 2018 for retinal edema lesions segmentation, and the results show that our proposed method achieves better segmentation accuracy with a high degree of reliability as compared to other state-of-the-art segmentation approaches. The code will be released on: https://github.com/LooKing9218/ReliableRESeg.

We represent the ResNeRF, a novel geometry-guided two-stage framework for indoor scene novel view synthesis. Be aware of that a good geometry would greatly boost the performance of novel view synthesis, and to avoid the geometry ambiguity issue, we propose to characterize the density distribution of the scene based on a base density estimated from scene geometry and a residual density parameterized by the geometry. In the first stage, we focus on geometry reconstruction based on SDF representation, which would lead to a good geometry surface of the scene and also a sharp density. In the second stage, the residual density is learned based on the SDF learned in the first stage for encoding more details about the appearance. In this way, our method can better learn the density distribution with the geometry prior for high-fidelity novel view synthesis while preserving the 3D structures. Experiments on large-scale indoor scenes with many less-observed and textureless areas show that with the good 3D surface, our method achieves state-of-the-art performance for novel view synthesis.